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ABSTRACT 


This thesis investigates whether the National Hurricane Center (NHC) 
operational product for producing probabilistic forecasts of tropical cyclone (TC) 
wind distributions could be further improved by examining the distributions of 
track errors it draws upon to calculate probabilities. The track spread/skill 
relationship for several global ensemble prediction system forecasts is examined 
as a condition for a description of a full probability distribution function. The 
2007, 2008, and 2009 NHC official track forecasts are compared to the ensemble 
prediction system model along-, cross-, and forecast-track errors. Significant 
differences in statistical properties were then identified among the groups to 
determine whether conditioning based on geographic location was warranted. 
Examination of each regional distribution interval suggests that differences in 
distributions existed for along-track and cross-track errors. Because errors for 
ensemble mean and deterministic forecasts typically have larger mean errors 
and larger variance than official forecast errors, it is unlikely that independent 
error distributions based on these models would refine the PDFs used in the 
probabilistic model. However, this should be tested with a sensitivity analysis 
and verified with the probability swath. Overall, conditional formatting suggests 
that the NHC probability product may be improved if the Monte Carlo (MC) model 
would draw from refined distributions of track errors based on TC location. 
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I. INTRODUCTION 


A. MOTIVATION 

Tropical cyclones (TCs) routinely affect society, the economy, the 
environment, and military operations. Although coastal counties in the 
contiguous United States account for only 17 percent of the total landmass, over 
50 percent of the population lives within this region (NOS 2010). Between 1960 
and 2008, the population of the U.S. coastline increased by 84 percent compared 
to a 69 percent increase in population for the entire country over the same time 
period. The Gulf of Mexico region alone has increased in population by 150 
percent from 1960 to 2008 (Wilson 2010). During the next several decades, 
these coastal regions are expected to exponentially increase in population, and 
with these new residents, comes an increase in vulnerability to severe weather 
induced by tropical cyclones. 

During the 2007-2009 hurricane seasons alone, 18 TCs either made 
landfall or came close enough to the U.S. coastline to become an evacuation or 
resource protection concern. Hurricane Ike, which made landfall in Cuba and the 
southeast Texas coast in September 2008, was the third costliest to hit the U.S. 
and the most substantial in deaths and destruction since Hurricane Katrina in 
2005. Fortunately, models handled this storm particularly well, with the accuracy 
of most official forecasts significantly better than average (NHC 2010k). 
However, improvements to TC forecast models could have reduced the amount 
of lives lost and property damaged during this storm and could so for storms in 
the future as well. 
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Figure 1. (a) Waves crash over the Galveston Seawall and the memorial for the 

1900 Galveston Hurricane before the arrival of Ike, (b) House on fire on 
Galveston Island as storm surge waters rise in advance of Ike, (c) 
Devastation on the Bolivar Peninsula due to storm surge from Ike, (d) 
Before and after image of the Bolivar Peninsula depicting the effects of 
storm surge, (e) Debris, boats, and trailers on the southbound lanes of I- 
45 heading toward Galveston, and (f) Ships waiting to enter Galveston 
Bay after Hurricane Ike (From NHC 2010k). 


Civilians along the U.S. coast are only part of the requirement for 
increasing tropical cyclone forecast accuracy. Military operations on land, sea, 
and air depend on accurate forecasts and adequate lead times to protect 
resources and ensure mission completion. A large number of military 
installations are along the Gulf Coast and Atlantic states from Naval Air Station 
(NAS) Kingsville, TX to NAS Key West, FL, to Portsmouth Naval Shipyard, ME. 

One installation that is highly vulnerable to TCs is Patrick Air Force Base, 
FL, where the 45**^ Weather Squadron (WS) is in charge of forecasting for 
launches at Kennedy Space Center and the Eastern Range at Cape Canaveral. 
Billions of dollars in equipment, facilities, and flight hardware, as well as the 
personnel and families who support the mission at these two sites, are vulnerable 
to the threats of TCs. Resource protection decisions made by NASA and Air 
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Force senior leadership rely on critical guidance from the 45th WS on track, 
storm size, and intensity based on NHC forecasts. 


a.) b.) 



Figure 2. Space shuttle (a) in process of rollback to Vehicle Assembly Building 

pictured in panel (b) (From Astroprof 2010). 

Particularly sensitive to hazardous weather associated with TCs is the 
space shuttle, which requires a rollback to the Vehicle Assembly Building (VAB) 
from the launch pad prior to the onset of sustained 40-knot winds. Planners 
require at least 48 hours to make a decision on initiating a rollback due to the 
time-consuming process of halting launch operations and transporting the shuttle 
back to safety (Winters 2006). Forecasters at the 45*'^ Weather Squadron rely on 
probabilistic forecasts to aid planners in making costly decisions such as a 
rollback. An overly cautious forecast resulting in a false alarm could cost 
hundreds of thousands of dollars and delayed scheduling of launches while a 
missed forecast could result in billions in dollars of damage to equipment or harm 
to personnel (Flauke 2006). 

B. OBJECTIVE 

Improving tropical cyclone track and intensity forecasts has long been the 
focus of the National Flurricane Center (NFIC) and other agencies. In the 1980s, 
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the National Weather Service (NWS) issued watch/warning graphical maps that 
included a white cone indicating the area where a TC was expected to verify 90% 
of the time. Improvements to this procedure by the NHC included strike 
probability forecasts in text format, which indicated the percentage of time a TC 
was expected to pass within 75 n mi to the right or 50 n mi to the left of a given 
point. A recent improvement is the use of a Monte Carlo (MC) model (DeMaria 
et al. 2009a) to calculate uncertainties in track, intensity, and wind radii. Hauke 
(2006) re-examined the track error distributions the MC model draws from by 
determining conditional distributions based on forecast confidence. An improved 
forecast ability resulted from this modification of the NHC operational wind 
probability program (DeMaria et al. 2009b). 

The objective of this thesis is to increase tropical cyclone wind distribution 
forecast accuracy to protect lives and resources of both military and civilian 
entities. For this thesis, the following hypothesis will be investigated: 

The refinement of current distributions of forecast track errors 
based on conditions derived from a specific parameter such as 
location would improve NHC and JTWC tropical cyclone 
probabilistic wind distribution forecasts. 

This hypothesis has particular relevance to the forecast support generated by the 
45**^ WS, but civilian and military entities worldwide can aid from improvements to 
TC track forecasts and accurate wind distributions in the warnings. 

This thesis will attempt to further improve TC forecasts by examining the 
distributions of track errors the NHC MC model draws upon to calculate 
probabilities. If it is possible to use different distributions for different situations, 
the value of the probabilistic output may be increased. Results of this particular 
study could lead to a reduction of the massive costs of overly cautious 
evacuations when forecast confidence is high, or even save lives by expanding 
the evacuation zone when forecast confidence is low. 
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Background material regarding important concepts related to this thesis is 
provided in Chapter II. The methodology used for this study is described in 
Chapter III. The results of the study are presented in Chapter IV, and the 
conclusions and future recommendations are given in Chapter V. 
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II. BACKGROUND 


A. NATIONAL WEATHER SERVICE PROBABILISTIC TROPICAL 

CYCLONE FORECASTS 

The TC forecast cone product (Figure 3) developed over a quarter-century 
ago depicts the NHC’s official track (solid black line for days 1-3 and dashed 
black line for days 45) with a shaded area (white for days 1-3 and stippled white 
for days 4-5) on either side indicating an approximate 67% probability of where 
the TC will likely go based on a sampling of track errors from the past five years 
(NHC 201 On). The forecast cone gives the general public a basic picture of 
where a TC is likely to go along with current watches and warnings along 
coastlines. This product was periodically updated with new track error statistics 
from its inception in 1983 until 2005, but relatively few changes have occurred in 
the product since then (DeMaria et al. 2009a). 



Figure 3. Flurricane Ike 5-day Watch/Warning/Forecast for 11 September 2008 
(From NHC 201 Og). The symbols are explained in the lower-right box, 
and an approximate distance scale for the cone is given at the top. 
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Since the beginning of the 2006 Atlantic and eastern North Pacific 
hurricane seasons, the NHC has been using tropical cyclone wind speed 
probability text and graphical products (NHC 201 Oo). Whereas previous 
forecasts focused more on single-track guidance, the goal of these new products 
has been to paint a more realistic picture of the actual weather that is forecast to 
occur at any given location. This probabilistic versus deterministic approach to 
forecasting allows decision-makers to better plan for the timing of weather events 
or for the likelihood that they will even occur at all (Wilks 2006). 


Storm-specific text products contain two sections as shown in Figures 4 
and 5. The Maximum Wind Speed (Intensity) Probability Table contains 
probabilities for maximum wind speeds at standard forecast hours for various 
intensity stages and for the five categories on the Saffir-Simpson Hurricane Scale 
(NHC 201 Ol). 
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Figure 4. Maximum wind speed (intensity) probability table for Hurricane Ike 

Advisory #42 (From NHC 2010m). 
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The wind speed probability table for specific locations in section II (Figure 
5) contains wind speed probabilities for selected coastal and inland cities along 
with each forecast issued by the NHC. Each wind speed probability text product 
provides probabilities for wind speeds of at least 34 kt (39 mph, tropical storm 
force, 50 kt (58 mph), or 64 kt (74 mph, hurricane force) at each listed location. 
Cumulative probabilities of occurrence and individual period probabilities are 
included in section II, e.g., the overall probability that the stated wind speed will 
occur at each location during the period between hour 0 and each listed forecast 
hour. Individual period probabilities of onset indicate the chances that the stated 
wind speed will start during each individual period at each location (NHC 20101). 
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Figure 5. Section II of the wind speed probability table for specific locations for 
Hurricane Ike Advisory #42 (From NHC 2010m). 


9 



Graphical probability products (Figure 6) identify regions where wind 
speeds of at least 34, 50, or 64 kt are expected to occur during cumulative time 
periods from 0-120-h in 12-h increments. These products represent where 
tropical storm force winds of at least 34 kt (Figure 6a) and hurricane force winds 
of at least 64 kt (Figure 6b) are expected occur for Ike. Drawing from operational 
forecast center track and intensity error distributions from the past five years, this 
Wind speed Probability Forecast Product (WPFP) uses a Monte Carlo (MC) 
method to generate 1000 realizations, or tracks, relative to a given forecast track 
(DeMaria et al. 2009a). 



Figure 6. (a) Tropical storm force wind speed probabilities and (b) hurricane 

force wind speed probabilities for Ike (2008) (From NFIC 201 Oh and 

2010i). 

These products take the focus off the expected center-line path of the eye 
of the storm and convey to users where winds of a certain magnitude may occur. 
Decision-makers can then make cost-benefit analysis decisions based on the 
probability of winds occurring at specific locations rather than relying on a single 
forecast cone representing one official forecast track. 

B. RECENT TC TRACK AND WIND DISTRIBUTION FORECAST 
IMPROVEMENT STUDIES 

Studies have been conducted on the performance, validity, and potential 
improvements of the NFIC Wind Probability Forecast Products (WPFP) 
introduced in 2006. Flauke (2006) examined distributions of forecast track errors 
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conditioned on forecast confidence to determine if significant differences exist in 
distribution characteristics. He used two predictors to define forecast confidence: 
the Goerss Predicted Consensus Error (GPCE, (Goerss 2000)) and the Global 
Forecast System (GFS) ensemble spread. The distributions of total-, along-, and 
cross-track errors from NHC official forecasts were defined for low, average, and 
high forecast confidence. Distributions of the GFS ensemble mean total-track 
errors were also defined for these three forecast confidence levels. Standard 
hypothesis testing methods were used to examine distribution characteristics. 
Using the GPCE values, Hauke (2006) found significant differences in nearly all 
track error distributions for each level of forecast confidence. The GFS ensemble 
spread did not provide a basis for statistically different distributions. Hauke 
(2006) concluded that these results suggest the NHC probability model would 
likely be improved if the MC model would draw from distributions of track errors 
based on the GPCE measures of forecast confidence. 

Shafer (2009) performed an objective evaluation of the WPFP 
performance and interpretation of the product for operational application, e.g., 
discerning which forecast probabilities represent low/moderate/high risk for the 
various wind speed and forecast interval categories. Shafer (2009) focused on 
four hurricane seasons from 2004-2007 and included all storms approaching 
areas of interest centered on Cocoa Beach, FL, Charleston, SC, New Orleans, 
LA, and Corpus Christi, TX. Verification statistics were computed for each of the 
21 forecast categories of the NHC probability product; three wind speed criteria 
(>= 34 kt, >= 50 kt, and >= 64 kt) and seven forecast time intervals (12, 24, 36, 
48, 72, 96, and 120-hours). Verifications included use of reliability and 
sharpness diagrams, as well as additional statistics designed to quantitatively 
measure the product performance. Shafer (2009) concluded that the WPFC 
performed well and within the acceptable range of skill, with the skill of the 
forecast system decreasing as forecast time interval increased. 
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Splitt et al. (2009) revised and expanded the Shafer (2009) study and 
found the NHC product has a tendency to over forecast in some 34 kt cases but 
sometimes under forecasts for the 60 kt cases. Results from the 50 kt cases 
were mixed but also exhibited a tendency to underforecast during the later 
intervals. 

Majumdar and Finocchio (2009) examined the ability of ensemble 
prediction systems to predict the probability that a tropical cyclone will fall within 
a certain area. Ensemble forecasts issued by the European Center for Medium- 
range Weather Forecasts (ECMWF) and the United Kingdom Meteorological 
Office (UKMET) of up to five days were evaluated for the 2008 Atlantic and 
western North Pacific seasons. The results demonstrated the potential for 
ensemble prediction systems to enhance probabilistic forecasts, and for the 
THORPEX Interactive Grand Global Ensemble (TIGGE) to be utilized by the 
operational and research communities. 

DeMaria et al. (2009) evaluated performance of the MC probability model 
in the Atlantic and Pacific basins for the 2006-2007 seasons and concluded that 
the model is relatively unbiased, and the forecasts are skillful using a Brier skill 
score and a relative operating characteristic score. DeMaria et al. (2009) also 
determined that basin-wide error statistics are a current limitation to the model 
and noted that it is possible to estimate the error of a given track forecast based 
upon the spread of a consensus of track models and other information known at 
the time the official forecast is made (Goerss 2007). 

Results of these studies during the past four years indicate the NFIC 
WPFP is an effective and beneficial forecast tool and that consensus models 
could enhance the performance of these products. Furthermore, conditioning on 
certain parameters such as forecast spread (Flauke 2006) may also improve the 
WPFP. Continued confirmation of the contributions of these products to 
increased effectiveness means continued research to improve these products is 
both worthwhile and necessary. 
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III. METHODOLOGY 


A. DATA 

1. Data Source 

The database for this study is from the 2007, 2008, and 2009 Atlantic 
hurricane seasons. With 40 named tropical storms and five unnamed tropical 
depressions, 1150 official forecasts were issued by the NHC during these three 
seasons. Considering there are seven time periods for each forecast (12, 24, 36, 
48, 72, 96, and 120-h), the number of potential track forecast errors is 8050 for 
all three years. By combining results from all three seasons, it is expected that 
this was a large enough data set to test the hypothesis of this thesis. 

While the 2005 season stands out as the most active on record, 
subsequent seasons have continued to influence civilian and military planners 
and operators. The 2007 season produced 14 named storms, which included six 
hurricanes, three of which reached major hurricane (Category 3 or higher) status 
(Figure 7). In addition, two tropical depressions also formed in the Atlantic. The 
numbers of hurricanes and major hurricanes were near the long-term averages 
for a season, but the number of named storms was slightly above average. For 
the first time since records began in 1851, two Category 5 hurricanes. Dean and 
Felix, made landfall during the 2007 season (NFIC 2010a). 

Dean made landfall as a Category 5 hurricane on the east coast of the 
Yucatan Peninsula and was responsible for about 40 deaths across the 
Caribbean, with the largest tolls in Mexico and Flaiti. Felix made landfall as a 
Category 5 hurricane in Nicaragua and was responsible for 130 deaths in 
Nicaragua and Flonduras as well as significant structural damage in Central 
America and the Caribbean Islands (NFIC 2010a). 
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Figure 7. Tracks for the 2007 Atlantic hurricane season. The track symbols are 
explained in the lower-left box and the storm names are listed in the 
upper-right box. (From NFIC 201 Od) 

The 2008 season (Figure 8) presented yet another challenge to 

forecasters, with an above-average 16 named storms in the Atlantic basin. Eight 

of these storms became hurricanes, and five strengthened into major hurricanes 

(NFIC 2010b). Flurricanes Gustav, Flanna, and Ike were the most notable TCs in 

terms of deaths, with 112, 500, and 102 occurring, respectively. Major wind and 

storm surge damage occurred during Gustav’s landfall in Cuba, and heavy rains 

in Flaiti caused destructive mudslides. Strong winds, high storm surges, and 

heavy rains also caused an estimated 4.3 billion dollars damage in Louisiana 

(NFIC 2010b). Flanna’s very heavy rainfall in Flaiti resulted in an estimated 500 

fatalities, and minor wind and flood damage occurred in the Turks and Caicos 

Islands. In the U.S., damage was relatively minor, but occurred over a large area 
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and totaled an estimated $160 million dollars (NHC 2010b). Hurricane Ike killed 
an estimated 74 people in Haiti and two people in the Dominican Republic with 
extensive wind and storm surge damage as it crossed the island of Cuba, where 
seven deaths were reported. Media reports indicated 19 direct TC-related 
deaths in Texas, Louisiana, and Arkansas. The remnants of Ike also caused 
wind damage and several dozen TC-indirect deaths across portions of the 
Mississippi and Ohio Valleys (NHC 2010b). 
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Figure 8. Tracks for the 2008 Atlantic hurricane season. The track symbols are 
explained in the lower-left box and the storm names are listed in the 
upper-right box. (From NHC 201 Oe) 
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While the 2009 season (Figure 9) had fewer storms, several different 
forecast scenarios occurred. The season contained nine named storms, three of 
which became hurricanes, and two of which became major hurricanes. There 
were also two tropical depressions that did not reach tropical storm strength 
(NHC 2010c). 
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Figure 9. Tracks for the 2009 Atlantic hurricane season. The track symbols are 
explained in the lower-left box and the storm names are listed in the 
upper-right box. (From NFIC 201 Of) 

The societal and physical impacts caused by each of these TCs during the 
2007, 2008, and 2009 seasons underscore the importance of adequately 
preparing for the potentially overwhelming impacts of TCs. Perhaps the most 
significant tool for adequate preparation is early and accurate track forecasting 
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with accurate wind distributions to warn people of the potential damage. This 
database from the 2007, 2008, and 2009 seasons will be used to test the basic 
hypothesis of the thesis. 

2. Data Format 

The data available from the NHC included every official forecast, a 
majority of the model forecasts, and all of the best-track positions of the TCs that 
occurred between 2007 and 2009. The A-Decks and B-Decks from the 
Automated Tropical Cyclone Forecast (ATCF) system as well as TIGGE 
(TFIORPEX (The Observing System Research and Predictability Experiment) 
Interactive Grand Global Ensemble) data were used in this thesis. The A-Decks 
are comprised of all the model and ensemble forecasts available to the NFIC 
during the season along with their official (OFC) forecasts. Information included 
in the A-Decks are the storm number, model, forecast time and period, forecast 
intensity, and forecast position in latitude and longitude (Flauke 2006). 

The B-Decks are the best-track positions of the Atlantic TCs in the 2007, 
2008, and 2009 seasons. The best-track position is the verifying position of the 
TC after all the information has been evaluated in post-storm analysis. Included 
in these files are the storm number, date and time, intensity, and verifying 
location in latitude and longitude. Thus, the A-Decks can be used in conjunction 
with the B-Decks to calculate track forecast error (Flauke 2006). 

Ensemble-based forecast tracks from three operational weather centers. 
United Kingdom Meteorological (UKMET), European Center for Medium-range 
Weather Forecasts (ECMWF), and National Centers for Environmental Prediction 
(NCEP), were obtained from the TIGGE data on the University Corporation for 
Atmospheric Research (UCAR) Computational and Information Systems 
Laboratory (CISL) Web site at http://dss.ucar.edu/datasets/ds330.3/MSS-file- 
list.html . These data are in a format similar to extensible Markup Language 
(XML) format, which is then labeled cyclone XML (CXML), and were converted 
for general use in statistical analysis and displays. 
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3. Ensuring Homogeneity of Data 

Track data obtained from the TIGGE Web site were compiled by groups of 
models, individual models, all regions, and individual regions. For example, 
comparisons of data were by all models (ECMWF+UKMET+GFS), by individual 
model (ECMWF, UKMET, and GFS) and by a combination of models 
(ECMWF+UKMET). One major challenge in this research was ensuring that all 
data were homogeneous before conditioning and conducting statistical analyses. 
When comparing these large datasets from several different ensemble models, it 
was critical to make sure that similar data were being compared in each group of 
files. For example, when grouped by all models and all regions, if the GFS did 
not have a control run for a specific date/time in 2007, but the ECMWF and 
UKMET did, then that case was eliminated. If the Along-Track Errors (ATE) for 
the ECMWF were not available for a certain time period, then they were 
eliminated as well. Each dataset, by virtue of being homogeneous, contained 
different amounts and types of data compiled from the TIGGE files. This resulted 
in a dataset that included homogenous model runs (OFC, control (CTL), 
deterministic (DET), and ensemble mean (EMN)) for the same forecast times (0- 
120-h), for the same errors (ATE, XTE, FTE), and for each group of models 
and/or regions. 

4. Ensemble Prediction Systems 

Different characteristics among the Ensemble Prediction Systems (EPS) 
used in this study are given in Table 1. Part of the problem behind ensuring a 
homogeneous dataset is the lack of standardization between ensemble models. 
For example, the ECMWF has 51 members and is run every 12-hours (00 UTC 
and 12 UTC) out to 384 hours, the UKMET has 23 members and is run every 12- 
hours (00 UTC and 12 UTC) out to 144 hours with intermediate runs (01 UTC 
and 13 UTC) out to 48 hours, and the GFS has 21 members and is run every 6 
hours (00 UTC, 06 UTC, 12 UTC, 18 UTC) out to 384 hours. For this research, 
the OOZ and 12Z runs were used in all cases. In each of these EPSs, the control 
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(CTL) model is typically the higher resolution deterministic (DET) model from that 
center, and then an even number ensemble model forecasts, at lower resolution 
are made from perturbed initial conditions. 

Table 1. Comparison of Ensemble Prediction Systems (EPS) examined in this 

study. 


EPS 

Members 

Frequency 

Duration 

ECMWF 

51 

00 UTC, 12UTC 

384 h 

UKMET 

23 

UTC (intermediate), 12 UTC, 13 UTC 
(intermediate) 

144 h (48 h) 

GFS 

21 

00 UTC, 06UTC, 12 UTC, 18 UTC 

384 


5. Forecast Position Errors 

The three types of errors (along-, cross-, and forecast-track) examined in 
this thesis are shown in Figure 10. The track error of the forecast is determined 
from the verifying best-track position. The forecast track error (FTE) is the great 
circle distance between the best-track and forecast positions. In addition, the 
cross-track error (XTE) and along-track error (ATE) are defined as illustrated in 
Figure 10. 



Figure 10. Definition of cross-track error (XTE), along-track error (ATE), and 
forecast track error (FTE). In this example, the forecast position is ahead 
of and to the right of the verifying best track position. Therefore, the XTE 
is positive (to the right of the best track) and the ATE is positive (ahead or 
faster than the best track) (From Tsui and Miller 1988). 
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6. Conditioning by Region 

Once all ensemble and official track forecast data were combined into 
homogeneous files, the procedure was then to begin conditioning these data 
based on certain parameters. Because location was the primary parameter to 
explore, the North Atlantic Basin was divided into six regions based on latitude 
and longitude (Figure 11). 



Figure 11. Geographic regions for conditioning Ensemble Prediction System 

(EPS) TC track errors and spread. 

B. STATISTICAL ANALYSIS METHODS 
1. Testing for Differences in Mean 

While analyses in this thesis are similar in scope to that of Flauke (2006), 
several differences will be described. In Figure 12, three distributions are 
examined. The two distributions in the low variability case (bottom row) have 
very little overlap and thus are significantly different. The distribution in the 
medium variability case (top row) has some overlap and is only slightly different. 
In the high variability case (middle row), it becomes more difficult to distinguish 
the two populations. 
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In this thesis, the first objective is to search for significant differences 
among the track forecast error distributions when conditioned on relative physical 
parameters. If only small differences exist as demonstrated by the high 
variability case in Figure 12, then there will likely be little improvement in the MC 
model by including that physical parameter. This is because the distributions are 
similar, and using them independently will not change the probability output 
(Hauke 2006). However, it is expected that construction of a conditional 
probability distribution when significant differences exist in the distributions that 
have low variability as in Figure 12 will then result in better discrimination in an 
MC model. 
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Figure 12. Three pairs of distributions with the same mean (From Trochim 2010). 

The objective method to determine if samples from two populations are 
significantly different is the t-test statistic (T) (Wilks 2006), which is a function of 
the differences between the two sample means and takes into account the sizes 
and variances of the distributions. The t-test statistic is defined as 


21 




















where -^i and -^2 are the means of the two samples, - 1^7 = 0 is the 

hypothesized difference between the two means. In Equation (1), and are 
the standard deviations of the two samples, and and ”z are the numbers of 
members in each sample. The t-statistic is evaluated using a .05 percent 
confidence level, which means the test will be in error no more than five out of 
100 times (Wilks 2006). 

The null hypothesis for this test is that the two means are the same (f^i - 

= 0) (Figure 13). If the null hypothesis is true, then the t-statistic will fall in the 
acceptance region of the t-distribution (t-statistic < t-critical). If the null 
hypothesis is false, then the t-statistic will fall in the critical region of the t- 
distribution (t-statistic > t-critical). 



Figure 13. Flypothesis test for differences in the means of two distributions. 

(From Wadsworth 2010). 
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As indicated in Figure 12, the means of two samples may be the same 
while the variances are different. This situation may apply with cross- and along- 
track forecast errors since the errors are both positive and negative and thus 
tend to cancel at times, which may then lead to very small differences between 
the distributions. However, the means of the distributions can be used as an 
indicator of the skewness. Using different distributions that have the same mean 
but different variances also makes them separable and may also improve the MC 
product (Hauke 2006). 

2. Histograms and Probability Distribution Function (PDF) Plots 

Quantitative results of this thesis are presented primarily in the form of 
histograms and probability distribution functions. The purpose of these visual 
representations of the data is to clearly and accurately convey useful information 
as a result of comprehensive data analysis. 

Several histograms and probability distribution functions are presented in 
this thesis to provide useful data summaries that convey the following 
information: the general shape of the frequency distribution (normal, chi-square, 
etc.), symmetry of the distribution or whether it is skewed, and modality 
(unimodal, bimodal, or multimodal). Frequency distribution histograms can be 
converted to a probability distribution using several methods. Dividing the tally in 
each group by the total number of data points defines the relative frequency. 
Then, maximum likelihood estimates (Wilks 2006) can be used to define the 
probability distribution function for a specific distribution form. The shape of the 
distribution conveys important information such as the probability distribution of 
the data (Netmba 2010). In this thesis, histograms and PDFs are used to 
determine the statistical character of forecast errors for each forecast interval. 
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IV. ANALYSIS AND RESULTS 


A. INTRODUCTION 

The goal of this thesis is to demonstrate the likely benefit to the probability 
model based on conditioning the track error distributions from which the MC 
model draws. The primary conditioning factor to be examined is TC location at 
the time a forecast is issued. If a storm is located in a particular region then it 
may be beneficial for the probabilistic model to draw from historic track errors 
that were produced in the same region. 

In addition to geographic regions, error distributions from specific model 
products are examined relative to the error distribution of the official forecast 
(OFC). For each EPS, the errors in ensemble mean (EMN) forecast are 
compared to the OFC track errors. The control forecast (CTL) track errors were 
not available for all models at all times. Therefore, track errors for the CTL 
forecasts are not used in the study. Flowever, the deterministic (DET) model 
forecast track errors are compared to the OFC and EMN forecast track errors. 
As defined in Chapter III, the homogeneous data are grouped based on model 
(all models, ECMWF, UKMET, ECMWF+UKMET, and GFS), by region (GOM, 
ECS, SBA, WCB, ECB, and MDR as in Figure 11), by error (ATE, XTE, and 
FTE), and by forecast interval (12, 24, 36, 48, 60, 72, 96, and 120-h). 

To determine whether conditioning based on geographic location was 
warranted, the homogeneous data groupings are examined to identify significant 
differences in statistical characteristics among the groups. The mean and 
standard deviations of ATEs, XTEs, and FTEs for EMN, DET, OFC, and CTL 
members of each data group are computed. Tests are then conducted to 
determine statistical differences among these errors in model and geographic 
groups. For combinations of model groups and geographic regions, the EMN 
errors were compared to OFC errors, the DET errors were compared to OFC 
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errors, and the EMN errors were compared to the DET errors. The probability 
distribution functions (PDFs) defined by ATEs and XTEs of EMN, OFC, and DET 
for all regions and all models were then created. 

B. STATISTICAL SIGNIFICANCE BY MODEL AND REGION 

Bar charts are used to compare the geographic regions, model groups, 
and errors. The abscissa represents the forecast interval from 0-120 hours and 
the ordinate represents statistical significance. A positive oriented bar identifies 
positive differences between respective errors that are considered to be 
statistically significant. A negative oriented bar identifies a negative difference 
that is statistically significant. 

1. Analysis and Results 

For the entire data set of all models and all regions, the ATEs, XTEs, and 
FTEs of the EMN are significantly greater than the OFC ATEs, XTEs, and FTEs 
for most forecast intervals at a significance level of 0.05 (Figure 14). The ATEs, 
XTEs, and FTEs of the DET are also significantly greater than OFC ATEs, XTEs, 
and FTEs for several forecast intervals. The smaller OFC forecast errors than 
both the EMN and DET forecasts indicates the NFIC forecasters are adding value 
to their guidance. The ATEs and FTEs of the EMN were greater than ATEs and 
FTEs of the higher resolution DET for shorter range forecasts between 12-60-h. 

Because one or more track forecast error types are significantly greater 
than the OFC forecast, it is worthwhile to investigate if any specific geographic 
region(s) contributes most to these errors (Figure 15). The significantly larger 
EMN errors than the OFC errors at 48-h and beyond in the All Regions group 
(Figure 14) seem to be due to errors from the ECB (Figure 15e), MDR (Figure 
15f), ECS (Figure 15b), and WCB (Figure 15d). Significantly greater EMN errors 
than OFC errors do exist in the GOM region, but primarily at forecast intervals 
between 12-60-h. Therefore, some evidence is found that the forecast track 
errors from the combination of all models EMN in different regions may contain 

additional information from only the OFC track errors. 
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In regional comparisons of the All Models DET and OFC track errors 
(Figure 14 middle panel), much more variability exists in that the significant 
differences in Figure 14 mainly come from the main development region (Figure 
15f, middle panel). Similarly, the differences between the All Models DET and 
EMN track errors over All Regions (Figure 14, bottom panel), are mainly from the 
main development region (Figure 15f, bottom panel). 

All Models, All Regions DET/EMN/OFC Comparison 



IMN.OfC j , . J I 1 

>17 >74 >X >4i >U >77 >«4 >K >100 >171 

hirvcast 



rof9ca9t titorvai 


Figure 14. Statistically significant (at a level of 0.05) differences in ATE, XTE, and 
FTE (see designations on the right side of the top panel) for the sample of 
all models and all regions. EMN and OFC errors (top), DET and OFC 
(middle), and DET and EMN (bottom) are given as a function of forecast 
interval (h). The positive (negative) bar values identify statistically 
significant differences as defined by the labels in each panel. 
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Figure 15. Statistically significant (at a level of 0.05) differences in ATE, XTE, and 
FTE (see designations on the right side of the top panel) for the sample of 
all models in (a) the Gulf of Mexico, (b) the east coast region, (c) the 
subtropical Atlantic, (d) the western Caribbean, (e) the eastern Caribbean, 
(f) and the main development region. EMN and OFC errors (top), DET 
and OFC (middle), and DET and EMN (bottom) are given as a function of 
forecast interval (h). The positive (negative) bar values identify statistically 
significant differences as defined by the labels in each panel. 
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Figure 15 Continued. 


For the combination of ECMWF and UKMET models, the EMN XTEs and 
FTEs in the WCB contribute most to All Models, All Regions errors in Figure 14, 
while few errors from the SBA contribute. The EMN FTEs and ATEs in the ECB 
and MDR contribute significantly to All Models, All Regions EMN errors (Figure 
15), and DET ATEs, XTEs, and FTEs contribute to All Models, All Regions DET 
errors. 

Because forecast availability for the GPS ensemble prediction system is 
less than for the ECMWF and UKMET, a second group was created by 
combining the ECMWF and UKMET models only (Figure 16). A similar pattern is 
evident in that the EMN FTEs are significantly larger than the OFC errors for 
forecast intervals greater than 48-h (Figure 16a, top panel). The DET track 
errors are also significantly greater than OFC track errors for intervals greater 
than 48-h (Figure 16, middle panel). There are not many significant differences 
between EMN and DET forecast errors (Figure 16a, bottom panel). 

Whereas it is clear that the majority of the significant error differences are 
in either the ATEs or FTEs, only rarely are significant differences found in the 
EMN, DET, and OFC XTEs. Consequently, the paths of the TCs are generally 

predicted without a bias, and the ATE is the major source of the FTE. 
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Figure 16. Statistically significant (at a level of 0.05) differences in ATE, XTE, and 
FTE (see designations on the right side of the top panel) for the 
combination of ECMWF and UKMET in (a) all regions, (b) the east coast 
region, and (c) the main development region. EMN and OFC errors (top), 
DET and OFC (middle), and DET and EMN (bottom) are given as a 
function of forecast interval (h). The positive (negative) bar values identify 
statistically significant differences as defined by the labels in each panel. 
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Comparison of the combined ECMWF and UKMET model forecasts for 
individual regions relative to all regions (Figure 16a) indicates that the significant 
errors are mainly from the ECS region (Figure 16b) and MDR (Figure 16c). 
Other regions (not shown) do not contain consistent track error distribution 
differences from all regions. 

Individual model forecasts are also examined for all regions and then 
compared to errors in each region. For the ECMWF forecasts only, significant 
differences exist between the EMN and the OFC errors at nearly every forecast 
interval for all regions (Figure 17a, top panel). Again, the significantly larger 
EMN errors are typically in the ATE and FTE values. Examination of the errors in 
the other regions (not shown) indicates that the majority of the significant EMN 
differences compared to OFC errors are for the eastern Caribbean (Figure 17b, 
top panel). 

The ECMWF DET errors also are significantly larger than the OFC errors 
for most forecast intervals (Figure 17a, middle panel). Flowever, these error 
differences do include XTE at shorter forecast intervals. Furthermore, no one 
region contributes to the differences defined in the All Regions group (not 
shown). It is noteworthy that no significant differences in even the XTEs exist 
between the ECMWF DET and the EMN in the ECB (Figure 17b, bottom panel). 
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Figure 17. Statistically significant (at a level of 0.05) differences in ATE, XTE, and 
FTE (see designations on the right side of the top panel) for the ECMWF 
in (a) all regions and (b) the eastern Caribbean. EMN and OFC errors 
(top), DET and OFC (middle), and DET and EMN (bottom) are given as a 
function of forecast interval (h). The positive (negative) bar values identify 
statistically significant differences as defined by the labels in each panel. 


Of all models, the UKMET had the fewest number of EMN and DET track 
errors (Figure 18a) that were significantly greater than OFC track errors when all 
regions are combined. Flowever, the EMN track forecast errors over the main 
development region are significantly greater than the OFC errors (Figure 18b, top 
panel). Also, the significant differences are only FTEs and ATEs, so again the 
path is consistently forecast, but the spread along the path contributes to the 
errors. There are also some significantly larger DET errors than OFC errors in 
the main development region (Figure 18b, middle panel). 
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Figure 18. Statistically significant (at a level of 0.05) differences in ATE, XTE, and 
FTE (see designations on the right side of the top panel) for the UKMET in 
(a) all regions and (b) main development region. EMN and OFC errors 
(top), DET and OFC (middle), and DET and EMN (bottom) are given as a 
function of forecast interval (h). The positive (negative) bar values identify 
statistically significant differences as defined by the labels in each panel. 


At almost all forecast intervals, the GFS EMN XTEs and FTEs were 
significantly greater than the OFC and DET errors for all regions (Figure 19a). 
The consistently larger GFS EMN XTEs and FTEs than the GFS DET errors 
(Figure 19a, bottom panel) indicates a significant degradation of the EMN tracks 
relative to the higher resolution DET model tracks. These errors primarily 
originated from the MDR (Figure 19b). Other regions (not shown) did not contain 
significantly large errors relative to All Regions. 
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Figure 19. Statistically significant (at a level of 0.05) differences in ATE, XTE, and 
FTE (see designations on the right side of the top panel) for the GFS in (a) 
all regions and (b) main development region. EMN and OFC errors (top), 
DET and OFC (middle), and DET and EMN (bottom) are given as a 
function of forecast interval (h). The positive (negative) bar values identify 
statistically significant differences as defined by the labels in each panel. 


2. Summary 

As expected, EMN track errors were frequently larger than OFC track 
errors for all regions and all models. The EMN track errors were frequently larger 
than DET errors, and each of these was greater than OFC track errors. 

The individual regions that contributed the most significant EMN and DET 
errors compared to the OFC errors for the All Models group were the ECB, MDR, 
and ECS. This error characteristic was also true for the ECMWF+UKMET group. 
For the ECMWF, the ECB region contributed the most to the error differences 
with the OFC forecasts. The MDR had the largest contribution to the UKMET 
forecast error differences and also to the GFS forecast error differences. 

A climatologically typical TC track from the Cape Verde Islands transits 
westward through the entire MDR, crosses part of the ECB, and then recurves 
northward and eastward through the ECS. The high frequency of storms during 
the 2007-2009 Atlantic hurricane seasons that moved along this typical TC track 
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was most likely the reason these three regions observed the most forecast 
errors. Fewer cases were available in the GOM, WCB, and SBA as fewer storms 
crossed these regions. However, the consistent contribution from MDR, ECB, 
and ECS regions in defining significant differences with OFC errors suggest that 
error distributions associated with these regions may be useful in defining 
probability distributions for tropical cyclone forecasts. 

C. PROBABILITY DENSITY FUNCTIONS BY MODEL AND REGION 

Based on the results of significant differences among model ensembles, 
deterministic, and official forecasts over all regions and individual regions, the 
probability distribution functions (PDF) of the forecast error distributions are 
analyzed. Significant differences in the PDF characteristics would imply that the 
probability model could be impacted by the use of these distributions. Because 
the majority of significant differences in Section B are in the ATEs, these errors 
are examined first. The PDFs of FTEs are not examined since only ATEs and 
XTEs are used in the probabilistic model. Only selected forecast intervals will be 
described in this chapter. The characteristics for the remaining forecast intervals 
are available in the Appendix. 

1. ATE Probability Distribution Functions 

Comparison of the EMN and OFC 12-h ATEs (Figure 20) indicates that 
the EMN error distribution in the ECB region has a much larger variance than the 
error distribution for the All Regions. Therefore, the EMN forecasts over the ECB 
have significantly different distributions than the ATEs in other regions with a 
standard deviation of 50 km for the All Regions (Figure 20a, box). By contrast, 
the probability distribution functions for the 12-h OFC forecasts (Figure 20b) in 
the All Regions and the individual regions are similar, with standard deviations 
ranging from 30 km to 59 km, which is again associated with the ECB region. 
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Figure 20. Normal probability distribution functions for the 12-h forecast ATEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative ATEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 


The PDFs for the 48-h EMN and OFC forecast ATEs are similar to the 12- 
h PDFs except that the magnitudes are larger (Figure 21). Notice the distinctly 
larger spread in the ECB region (Figure 21a) relative to the All Regions and the 
other regions. For these 48-h forecasts, the standard deviation in the ECB is 225 
n mi versus 136 n mi for the All Regions and a range from 76 km to 150 km for 
the other regions. Whereas the PDFs for the 48-h OFC forecasts in the All 
Regions and all other regions besides the ECB region are quite similar (Figure 
21b), the OFC PDF for the ECB region is distinct. Indeed, the OFC standard 
deviation in this region is almost the same as for the 48-h EMN (209 km versus 
225 km). 
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Figure 21. Normal probability distribution functions for the 48-h forecast ATEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative ATEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 


Beyond 48-h, the PDFs have larger spreads and increases in standard 
deviations of the EMN and the OFC ATEs (Figure 22). The ECS region has 
significantly less variability than the All Regions PDF of OFC ATEs (Figure 22b). 
Note the large EMN spread in the GOM. This could be a result of the usually 
slow pace of storms in this region and the associated difficulty in forecasting the 
path at the -i-84-h forecast interval. 
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Figure 22. Normal probability distribution functions for the 84-h forecast ATEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative ATEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 


At -i-120-h, the ECB, MDR, and ECS regions have smaller standard 
deviations about the EPS mean ATE than for the All Regions. Indeed, the ECB 
and MDR have ATE standard deviations that are less than half of the standard 
deviation of the All Region ATEs. These small standard deviations indicate a 
small variability in the storm translation speeds in these regions even at five 
days, which probably arises due to the African Easterly waves having consistent, 
predictable paths and translation speeds. The ECS and MDR regions also have 
significantly less variability than the all regions PDF of OFC ATE errors (Figure 
23b). Recall that these regions also contributed most to the significantly larger 
EMN track errors compared to the OFC track errors. 
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While the results of the t-test analyses for the all models and all regions 
groups in Chapter IV.B indicates significant differences between EMN and OFC 
beyond 48-h, the PDFs of ATEs do not become significantly 
different in terms of variance until 84-h and beyond. Therefore, it appears that 
conditioning for the ATE in these regions for the all model group would not 
improve the MC probability model until beyond the 72-h forecast interval. 
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Figure 23. Normal probability distribution functions for the 120-h forecast ATEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative ATEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 


2. XTE Probability Distribution Functions 

Not many significant differences were found between the mean EMN and 
OFC XTEs (Figures 14 and 15) as were found between the EMN and OFC ATEs. 
For example, recall that the variance of the EMN XTE All Regions group at 24-h 
is larger than the All Regions group of the OFC XTE distribution (Figure 14). 
However, some differences were found in the PDF characteristics for the EMN 
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and OFC XTEs. It is clear in Figure 24a that the PDFs associated with XTEs in 
the 24-h EMN forecasts of the All Models groups begins to vary much more than 
the OFC XTE distributions (Figure 24b). Flowever, the variance in the EMN XTE 
PDF for the ECB region does not increase as rapidly with forecast interval as the 
variance in ATE (Figure 20a). 

By 120-h, many of the probability distribution functions of the OFC XTEs 
for several of the individual regions have less variance than the All Regions 
group (Figure 25b). Flowever, the main contribution to the larger variance in the 
OFC XTE All Regions group comes from the MDR, which has a larger negative 
bias and a larger variance. The PDFs of EMN XTEs at 120-h have means and 
variances that are quite similar (Figure 25a), even though the mean differences 
between EMN XTE and OFC XTE were not significantly different for the All 
Regions group (Figure 14) or for any regional group (Figure 15). The variances 
are significantly larger for the ECB and ECS region. 
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Figure 24. Normal probability distribution functions for the 24-h forecast XTEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative XTEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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The GOM region was the only region in which the OFC XTE variance was 
significantly larger than the EMN variance (Figure 24). The GOM was the only 
region in which the OFC XTE variance was larger than the variance in All 
Regions. 
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Figure 25. Normal probability distribution functions for the 120-h forecast XTEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative XTEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 


3. Testing for Sensitivity 

Testing the sensitivity of various error distribution characteristics is 
necessary to determine which distributions improve the Wind Speed Probability 
Forecast Products created by DeMaria et al. (2009). The test product should be 
similar in framework, but modified slightly. Error distributions would be defined 
that are associated with ATEs and XTEs of for selected forecast combinations 
and regions, such as the official tracks. All Models, ECMWF and UKMO 
combined, and ECMWF, UKMO, and GFS individually. Differences in 


41 











probabilities could be evaluated among groups and regions, and these 
distributions could be evaluated to fit various functional forms. Differences 
related to distribution characteristics could then be tested, and once the “best” fit 
is defined, then serial correlation could be identified. The next step would be to 
evaluate differences among groups and regions and define residual distributions 
for sampling. Once a sampling technique is determined, then intensity estimates 
via linear model fit could be applied, and serial correlation and track-intensity 
dependence could be accounted for. The last step would be to apply the wind 
distribution model and define probabilities. 

4. Summary 

Analysis of the PDF characteristics associated with the All Models group 
has resulted in identification some basic differences among the distributions of 
XTEs and ATEs of the EMN and OFC track forecasts. As a first approximation, 
the normal distribution has been fit to the two error measures. This choice is 
based on analysis of DeMaria et al. (2009) and Majumdar and Finnochio (2010) 
in which normal distributions were used to define ATE and XTE characteristics 
associated with OFC and EMN forecasts. The All Models group was examined 
because this group provided the most robust comparison of ensemble systems to 
compare with the OFC forecasts (Figures 14 and 15). Furthermore, it was 
anticipated that a combination of models would more likely yield a normal 
distribution of ATEs and FTEs. 

As defined by DeMaria et al. (2009), the character of the error distribution 
from which the Monte Carlo method samples will determine the probability swath 
for distribution of threshold wind values such as tropical storm/gale or hurricane 
force. As Flauke (2006) showed, conditioning on expected forecast difficulty 
produced distributions such that the variability decreased with increased forecast 
confidence. Therefore, the probability swath was shown to be narrower in high- 
confidence forecasts. The hypothesis being investigated in this thesis is that 
additional refinement of key probability distributions could be gained by 
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conditioning on the individual six regions. The distributions defined in Figures 
14-26 do demonstrate that statistically significant differences exist between EMN, 
DET, and OFC track forecasts. In most cases, the EMN mean ATEs and XTEs 
are significantly larger than OFC errors. Furthermore, the DET errors are larger 
than the OFC errors but less than the EMN errors. Therefore, it appears that 
using forecast track error distributions based on EMN or DET forecasts would not 
sharpen the distributions from which the Monte Carlo sampling is drawn. 

Additionally, the normal PDFs of EMN ATE and XTE, and OFC ATE and 
XTE do indicate a large variation in variance for the distributions. Distributions 
defined by the DET errors (not shown) are similar to those for the EMN errors. 
To determine if conditioning by region would refine the PDFs for the MC 
probability model, the variances between OFC and EMN error distributions are 
examined. The F-test for equal variances between two probability distributions 
was used to examine whether differences in variances exhibited in Figures 20- 
25 are significantly different. In particular, it is important to identify which regions 
have error distributions that have significantly less variance than the All Regions 
group for the OFC and EMN forecasts. 

Comparison of variances among OFC forecast XTE distributions from 
individual regions indicates significantly less variance in forecast distributions 
from the WCB and ECS regions compared to the All Regions group (Table 2). A 
significant difference in variance is not found between the MDR and All Regions 
distributions or the ECB and all region distributions. There is less variance in the 
GOM XTE distribution than the OFC XTE distribution, but this is only true for 
short-range forecasts (Table 2). At longer forecast intervals, the OFC XTE 
variance for all regions is less than that for the GOM region only. 
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Table 2. Results of the F-test for the differences in variances between the All 
Models/All Regions group and All Models/individual regions groups of 
OFC XTEs. A shaded (hatched) cell indicates that the variances are 
significantly different and the All Regions group variance is less (more) 
than the individual region variance. 
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Table 3. Results of the F-test for the differences in variances between the All 
Models/All Regions group and All Models/individual regions groups of 
OFC ATEs. A shaded (hatched) cell indicates that the variances are 
significantly different and the All Regions group variance is less (more) 
than the individual region variance. 
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The character of the variance in ATE PDFs (Table 3) is fundamentally 
different from the XTE distribution variances (Table 2). The distributions 
conditioned on individual regions have smaller variances in ATEs at short 
forecast ranges over the GOM and WCB regions. At longer forecast intervals, 
the variances from distributions defined from the MDR and ECS regions become 
less than the OFC ATE errors from the All Regions, which is currently used in the 
MC probability model (DeMaria et al. 2009). 

The collections of XTEs and ATEs for the All Regions group do not fit a 
normal distribution at any forecast interval (first columns in Tables 4 and 5, 
respectively. For individual regions and XTEs, only shorter range forecasts over 
the ECS are significantly non-normal. However, nearly all distributions of ATEs 
are non-normal (Figure 26). 

It is possible that the relatively small dataset limited by the TIGGE data 
availability causes the error distributions to be non-normal. However, 
examination of other distributions of ATEs, e.g.. Figure 26a, indicates that most 
contain extreme values. Therefore, an extreme-value type of distribution may be 
more appropriate for fitting to forecast track errors. 
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Table 4. Results of the chi-square tests for goodness of fit of the OFC XTEs to a 
normal distribution. Shaded cells define forecast interval and region for 
which the hypothesis of a good fit to a normal distribution is rejected at the 
0.05 level. Cells with vertical lines define forecast intervals and regions for 
which the data sample was too small to meet cell population requirements 

for the chi-square test. 


Chi-Square Goodness of Fit for OFC XTFs to a Normal Distribution 



Table 5. Results of the chi-square tests for goodness of fit of the OFC ATEs to a 
normal distribution. Shaded cells define forecast interval and region for 
which the hypothesis of a good fit to a normal distribution is rejected at the 
0.05 level. Cells with vertical lines define forecast intervals and regions for 
which the data sample was too small to meet cell population requirements 

for the chi-square test. 
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Figure 26. Distribution of ATEs (a) and XTEs (b) for all models, all regions at the 

+84-h forecast. 

It is important to examine how appropriate a normal distribution as a base 
PDF is for use in probability models. As an example, a normal PDF is fit to 84-h 
EMN ATE and XTE errors and displayed in Figure 26. It is clear that the 
distribution of ATE errors does not fit a normal distribution while the distribution of 
XTEs seems to fit the normal distribution quite well. To examine the use of the 
normal distribution for OFC track error distributions defined above in the variance 
comparison, a chi-square goodness of fit test is applied to the normal PDFs and 
OFC XTE and ATE data (Tables 4-5). 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 

This study was the first step in an investigation of whether the MC 
probability model would be improved by conditioning the TC track error 
distribution on storm location rather than on basin-wide error statistics. If it is 
possible to use different track error distributions for different storm locations, the 
probability wind output may have more utility to decision makers. The goal is to 
then improve TC track forecasts that ultimately result in decreasing societal and 
physical impacts to both civilian and military entities. 

The initial step was to test three seasons of ensemble model track error 
distributions (EMN and DET) to see if they are significantly different from the 
official (OFC) track error distributions. The homogeneous data were grouped 
based on individual models, groups of models, and by region. Significant 
differences in statistical characteristics were then identified among the groups to 
determine whether conditioning based on geographic location was warranted. 
Once the means and standard deviations of all errors for each group of data 
were computed, tests were then conducted to determine statistical differences 
among the error distributions in model and geographic groups. The EMN, OFC, 
and DET errors were intercompared for combinations of model groups and 
geographic regions. 

Examination of each regional distribution for each forecast interval 
suggests that differences in distributions existed for ATEs and XTEs. Flowever, 
these differences vary by forecast interval and regions. Because errors for EMN 
and DET forecasts typically have larger mean errors and larger variances than 
OFC forecast errors, it is unlikely that independent error distributions based on 
these models would refine the PDFs used in the probabilistic model. However, 
this should be tested with a sensitivity analysis and verified with a probability 
swath. It is possible that a combination of PDFs from the OFC and the EMN 
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forecasts would provide a more complete probabilistic representation of the wind 
distributions. Although no consistent patterns were found with respect to 
variance comparisons between ATE and XTE OFC error distributions over all 
regions and individual regions, enough evidence was presented that warrants 
testing of conditional distributions based on region in the MC probability model. 

Finally, the lack of fit to a normal distribution associated with most of the 
probability distributions suggests that alternative distribution functions may 
improve the probability model. However, the relatively small sample of only three 
years available to this study may adversely impact the tests of distribution fit. 
Additional sensitivity testing in a MC probability model is required to determine 
how these distributions would impact the WPFP. 

Based on the results of this study, it is expected that the refinement of 
current distributions of forecast track errors based on conditions derived from 
specific parameters such as ensemble spread or storm location should improve 
NHC tropical cyclone probabilistic wind distribution forecasts. 

B. RECOMMENDATIONS 

The next step should be to change the MC model code to draw from 
along-track and cross-track error distributions based on region instead of just one 
error distribution for all track forecast situations and regions. The model should 
then be tested to see if the probabilistic wind distribution accuracy is significantly 
improved. If so, the operational MC model should be modified to include these 
additional parameters. 

Future research should concentrate on other factors that may influence 
historic track error distributions. Some of these factors may be steering flow 
characteristics, storm intensity, and weather regime of the eastern U.S. 
Historical TIGGE data are also available for the western North Pacific basin, and 
similar parameters could be explored for a multitude of seasons and storms. 
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APPENDIX. DET/EMN/OFC COMPARISONS 
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Figure 27. Statistically significant (at a level of 0.05) differences in ATE, XTE, and 
FTE (see designations on the right side of the top panel) for the GFS in (a) 
Gulf of Mexico and (b) east coast, (c) subtropical Atlantic, (d) western 
Caribbean, and (e) eastern Caribbean. EMN and OFC errors (top), DET 
and OFC (middle), and DET and EMN (bottom) are given as a function of 
forecast interval (h). The positive (negative) bar values identify statistically 
significant differences as defined by the labels in each panel. 
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Figure 27 Continued. Statistically significant (at a level of 0.05) 
differences in ATE, XTE, and FTE (see designations on the right side of the top 
panel) for the GFS in the (e) eastern Caribbean. EMN and OFC errors (top), 
DET and OFC (middle), and DET and EMN (bottom) are given as a function of 
forecast interval (h). The positive (negative) bar values identify statistically 
significant (not significant) differences as defined by the labels in each panel. 
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Figure 28. Normal probability distribution functions for the 24-h forecast ATEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative ATEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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Figure 29. Normal probability distribution functions for the 36-h forecast ATEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative ATEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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Figure 30. Normal probability distribution functions for the 60-h forecast ATEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative ATEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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Figure 31. Normal probability distribution functions for the 72-h forecast ATEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative ATEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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Figure 32. Normal probability distribution functions for the 96-h forecast ATEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative ATEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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Figure 33. Normal probability distribution functions for the 108-h forecast ATEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative ATEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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Figure 34. Normal probability distribution functions for the 12-h forecast XTEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative XTEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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Figure 35. Normal probability distribution functions for the 36-h forecast XTEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative XTEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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Figure 36. Normal probability distribution functions for the 48-h forecast XTEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative XTEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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Figure 37. Normal probability distribution functions for the 60-h forecast XTEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative XTEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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Figure 38. Normal probability distribution functions for the 72-h forecast XTEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative XTEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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Figure 39. Normal probability distribution functions for the 84-h forecast XTEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative XTEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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Figure 40. Normal probability distribution functions for the 96-h forecast XTEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative XTEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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Figure 41. Normal probability distribution functions for the 108-h forecast XTEs by 
the (a) EMN and (b) OFC for the group of All Models. The abscissa is the 
positive or negative XTEs in kilometers, and the ordinate represents 
frequency of occurrence. Curves are color-coded by region (see box) with 
the mean, standard deviation, and number of members shown in the 

legend. 
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